Positional, rotational and scale invariant optical correlation method and apparatus

ABSTRACT

A method and electro-optical apparatus for correlating two functions, f 1  (x,y) and f 2  (x,y) which are shifted, scaled and rotated versions of each other without loss of signal-to-noise and signal-to-clutter ratios as compared to the autocorrelation case. The coordinates of two correlation peaks provide an indication of the scale and orientation differences between the two functions. In performing the method, the magnitudes of the Fourier transforms of the functions are obtained, |F 1  (ω x , ω y )| and |F 2  (ω x ,ω y )| and then a polar coordinate conversion is performed, and the resultant functions F 1  (r,θ) and F 2  (r,θ) are logarithmically scaled in the r coordinate. The functions thus produced F 1  (e.sup.ρ,θ) and F 2  (e.sup.ρ,θ) are Fourier transformed to produce the Mellin transforms M 1  (ω.sub.ρ, ω.sub.θ) and M 2  (ω.sub.ρ,ω.sub.θ). The conjugate of one of these Mellin transforms is obtained, and the product of this conjugate with the other Mellin transform is produced and, subsequently, Fourier transformed to complete the correlation process.

The present invention relates generally to optical pattern recognitionsystems, and, more particularly, to optical correlation apparatus andmethods utilizing transformations that are shift, scale and rotationallyinvariant.

In the correlation of 2-D information, the signal-to-noise ratio of thecorrelation peak decreases significantly when there are scale androtational differences in the data being compared. For example, in onecase of a 35 mm transparency of an aerial image with about 5 to 10lines/mm resolution, this ratio decreased from 30 db to 3 db with a 2percent scale change and a similar amount with a 3.5° rotation.

Several methods have been advanced for overcoming the signal lossesassociated with the scale, shift and rotational discrepanciesencountered in optical comparison systems. One proposed solutioninvolves the storage of a plurality of multiplexed holographic spatialfilters of the object at various scale changes and rotational angles.Although theoretically feasible, this approach suffers from a severeloss in diffraction efficiency which is proportional to the square ofthe number of stored filters. In addition, a precise synthesis system isrequired to fabricate the filter bank, and a high storage densityrecording medium is needed.

A second proposed solution involves positioning the input behind thetransform lens. As the input is moved along the optic axis the transformis scaled. Although useful in laboratory situations, this method is onlyappropriate for comparatively small scale changes, i.e., 20 percent orless. Also, since this method involves mechanical movement ofcomponents, it cannot be employed in those applications where theoptical processor must possess a real time capability.

Mechanical rotation of the input can, of course, be performed tocompensate for orientation errors in the data being compared. However,the undesirable consequences of having to intervene in the opticalsystem are again present.

In applicants' co-pending application, Ser. No. 707,977, filed July 231976, there are disclosed correlation methods and apparatus which useMellin transforms that are scale and shift invariant to compensate forscale differences in the data being compared. The systems thereindisclosed, however, do not compensate for orientation errors in thisdata.

It is, accordingly, an object of the present invention to provide atransformation which is invariant to shift, scale and orientationalchanges in the input.

Another object of the present invention is to provide an opticalcorrelation method and apparatus for use with 2-D data having shift,scale and rotational differences.

Another object of the present invention is to provide a method ofcross-correlating two functions which are scale and rotated versions ofone another where the correlation peak has the same signal-to-noiseratio as the autocorrelation peak.

Another object of the present invention is to provide an electro-opticcorrelator whose performance is not degraded by scale and orientationaldifferences in the data being compared and which provides informationindicative of the magnitudes of these differences.

Other objects, advantages and novel features of the invention willbecome apparent from the following detailed description of the inventionwhen considered in conjunction with the accompanying drawings wherein:

FIG. 1 is a block diagram illustrating a positional, rotational andscale invariant transformation system;

FIG. 2 is a block diagram illustrating the real time implementation ofthe transformation of FIG. 1;

FIG. 3 shows the sequence of operations carried out in thecross-correlation method of the present invention;

FIG. 4 shows a correlation configuration for practicing the method ofFIG. 3; and

FIG. 5 shows the correlation peaks appearing in the output plane of thecorrelator of FIG. 4.

The present invention provides a solution for the shift, scale androtational differences between the input and reference data by utilizinga transformation which is itself invariant to shift, scale andorientational changes in the input. As shown in FIG. 1, the first stepin the synthesis of such a transformation is to form the magnitude ofthe Fourier transform |F(ω_(x),ω_(y))| of the input function f(x,y).This eliminates the effects of any shifts in the input and centers theresultant light distribution on the optical axis of the system.

Any rotation of f(x,y) rotates |F (ω_(x),ω_(y))| by the same angle.However, a scale change in f(x,y) by "a" scales |F(ω_(x),ω_(y))| by 1/a.

The effects of rotation and scale changes in the light distributionresulting from the Fourier transform of f (x,y) can be separated byperforming a polar transformation on |F(ω_(x),ω_(y))| from (ω_(x),ω_(y))coordinates to (r,θ) coordinates. Since θ = tan⁻¹ (ω_(y) /ω_(x)) and r =(ω_(x) ² +ω_(y) ²)^(1/2), a scale change in |F| by "a" does not affectthe θ coordinate and scales the r coordinate directly to r = ar.Consequently, a 2-D scaling of the input function is reduced to ascaling in only one dimension, the r coordinate, in this transformedF(r,θ) function.

If a 1-D Mellin transform in r is now performed on F'(r, θ), acompletely scale invariant transformation results. This is due to thescale invariant property of the Mellin transform.

The 1-D Mellin transform of F(r, θ) in r is given by ##EQU1## where ρ =ln r. The Mellin transform of the scaled function F" = F(ar,θ) is then

    M' (ω.sub.ρ,θ) = a.sup.-jω ρM(ω.sub.ρ,θ)

from which the magnitudes of the two transforms are seen to beidentical. One arrangement for optically implementing the Mellintransform is disclosed in applicants' co-pending application,above-identified, and there it is shown that ##EQU2## where ρ = ln r.From equation (3), it can be seen that the realization of the requiredoptical Mellin transform simply requires a logarithmic scaling of the rcoordinate followed by a 1-D Fourier transform in r. This follows fromequation (3) since M(ω.sub.ρ,θ) is the Fourier transform of F(expρ,θ).

The rotation of the input function f(x,y) by an angle θ₀ will not affectthe r coordinate in the (r,θ) plane. If, for example, the inputF(ω_(x),ω_(y)) is partitioned into two sections F₁ (ω_(x),ω_(y)), F₂(ω_(x),ω_(y)) where F₂ is a segment of F that subtends an angle θ₀, theeffects of a rotation by θ₀ is an upward shift in F₁ (r,θ) by θ₀ and adownward shift in F₂ (r,θ) by 2π - θ₀. Thus, while the polartransformation has converted a rotation in the input to a shift in thetransform space, the shift is not the same for all parts of thefunction.

These shifts in F(r,θ) space due to a rotation in the input can beconverted to phase factors by performing a 1-D Fourier transform onF(r,θ).

The final Fourier transform shown in FIG. 1 is a 2-D transform in whichthe Fourier transform in ρ is accomplished to effect scale invariance bythe Mellin transform and the Fourier transform in θ is used to convertthe shifts due to θ₀ to phase terms. The resultant function is, thus, aMellin transform in r, and, hence, it is denoted by M in FIG. 1.

If the complete transformation of f(x,y) is represented by

M (ω.sub.ρ,ω.sub.θ) = m₁ (ω.sub.ρ,ω.sub.θ) + m₂ (ω.sub.ρ,ω.sub.θ) (4)

the transformation of the function f'(x,y), which is scaled by "a" androtated by θ₀ is given by

    M'(ω.sub.ρ,ω.sub.θ) = M.sub.1 (ω.sub.ρ,ω.sub.θ) exp[-j(ω.sub.ρ lna+ω.sub.ρ θ.sub.0)]                     (5)

+ M₂ (ω.sub.ρ,ω.sub.θ) exp{-j[ω.sub.ρ lna-ω.sub.θ (2π-θ₀)]}

The positional, rotational and scale invariant (PRSI) correlation isbased on the form of equations (4) and (5). If the product M*M' isformed, we obtain

    M*M' = M*M.sub.1 exp[-j(ω.sub.ρ lna+ω.sub.θ θ.sub.0)]                                           (6)

+M*M₂ exp{-j[ω.sub.ρ lna-ω.sub.θ (2π-θ₀)]}

The Fourier transform of equation (6) is

    f.sub.1 * f * δ(ρ'-lna) * δ(θ'-θ.sub.0) + f * f.sub.2 * δ(ρ'-lna) * δ(θ'+2π-θ.sub.0) (7)

The δ functions in equation (7) identify the locations of thecorrelation peaks, one at ρ' = ln a, θ' = θ₀ ; the other at ρ' = ln a,θ' = (2π+θ₀). Consequently, the ρ' coordinate of the peaks isproportional to the scale change "a" and the θ' coordinate isporportional to the rotational angle θ₀.

The Fourier transform of equation (6) thus consists of two terms:

(a) the cross-correlation F₁ (expρ,θ) * F(expρ,θ) located, as indicatedabove, at ρ' = ln a and θ' = θ₀ ;

(b) the cross-correlation F₂ (expρ,θ) F(expρ,θ) located at ρ' = ln a andθ' = (2π+θ₀), where the coordinates of this output Fourier transformplane are (ρ',θ').

If the intensities of these two cross-correlation peaks are summed, theresult is the autocorrelation of F(expρ,θ). Therefore, thecross-correlation of two functions that are scaled and rotated versionsof one another can be obtained. Most important, the amplitude of thiscross-correlation will be equal to the amplitude of the autocorrelationfunction itself.

Referring now to FIG. 2, which illustrates one electroopticalarrangement for implementing the positional, rotational and shiftinvariant transformation, the input f(x,y), which may be recorded on asuitable transparency 20 or available in the form of an appropriatetransmittance pattern on the target of an electron-beam-addressedspatial light modulator of the type described in the article,"Dielectric and Optical Properties of Electron-Beam-Addressed KD₂ PO₄ "by David Casasent and William Keicher which appeared in the December1974 issue of the Journal of the Optical Society of America, Volume 64,Number 12, is here illuminated with a coherent light beam from asuitable laser not shown and Fourier transformed by a spherical lens 21.A TV camera 22 is positioned in the back focal plane of this lens andarranged such that the magnitude of the Fourier transform[F(ω_(x),ω_(y))] constitutes the input image to this camera. As is wellknown, camera 22 has internal control circuits which generate thehorizontal and vertical sweep voltages needed for the electron beamscanning, and these waveforms are extracted at a pair of outputterminals as signals ω_(x) and ω_(y). The video signal developed bysanning the input image is also available at a third output terminal.

Horizontal and vertical sweep voltages ω_(x) and ω_(y) are subject tosignal processing in the appropriate circuits 23 and 24 to yield thequantities (1/2) ln (ω_(x) ² +ω_(y) ²) and tan⁻¹ (ω_(x) /ω_(y)),respectively. It will be recalled that the results of this signalprocessing, which may be performed in an analog or digital manner, isthe polar coordinate transformation of the magnitude of the Fouriertransform of the input function and its subsequent log scaling in r.

The function F(e.sup.ρ,θ) is formed on the target of an EALM tube of thetype hereinbefore referred to. In this regard, the video signal fromcamera 22 modulates the beam current of this tube while the voltagesfrom circuits 23 and 24 control the deflection of the electron beam.Instead of utilizing an electron-beam-addressed spatial light modulator,an optically addressed device may be used wherein the video signalmodulates the intensity of the laser beam while deflection system 25controls its scanning motion. It would also be mentioned that thetransformation can also be accomplished by means of computer generatedholograms.

The function M(ω.sub.ρ,ω.sub.θ) is obtained by Fourier transformingF(e.sup.ρ,θ) and this may be accomplished by illuminating the target ofthe EALM tube with a coherent light beam and performing a 2-D Fouriertransform of the image pattern.

FIG. 3 shows the sequence of steps involved in correlating two functionsf₁ (x,y) and f₂ (x,y) that differ in position, scale and rotation. Itwould be mentioned that this method may be implemented by optical ordigital means. Thus, all of the operations hereinafter set forth may beperformed with a digital computer. However, the following descriptioncovers the optical process since it has greater utility in real timeoptical pattern recognition systems.

The first step of a method is to form the magnitude of the Fouriertransform of both functions |F₁ (ω_(x),ω_(y))} and |F₂ (ω_(x),ω_(y))}.This may be readily accomplished, as is well known, with a suitable lensand an intensity recorder with γ = 1. Next, a polar coordinateconversion of these magnitudes is performed to produce F₁ (r,θ) and F₂(r,θ). The r coordinate of these functions is now logarithmically scaledto form F₁ (expρ,θ) and F₂ (expρ,θ). A second Fourier transform iscarried out to produce the Mellin transform of F(r,θ) in r and theFourier transform in θ. The resultant functions being M₁(ω.sub.ρ,ω.sub.θ) and M₂ (ω.sub.ρ,ω.sub.θ). The conjugate Mellintransform of F(r,θ) which is M₁ *(ω.sub.ρ,ω.sub.θ) is formed and, forexample, recorded as a suitable transparency. This can be readilyaccomplished by conventional holographic spatial filter synthesismethods which involve Fourier transforming F₁ (expρ,θ) and recording thelight distribution pattern produced when a plane wave interferes withthis transformation.

The correlation operation involves locating the function F₂ (expρ,θ) atthe input plane of a conventional frequency plane correlator andpositioning the conjugate Mellin transform recording M₁*(ω.sub.ρ,ω.sub.θ) at the frequency plane. The light distributionpattern leaving the frequency plane when the input plane is illuminatedwith a coherent light beam has as one of its terms M₁ *M₂ and thisproduct when Fourier transformed completes the cross-correlationprocess. The correlation of the two input functions in this methodappears as two cross-correlation peaks, and the sum of their intensitiesis equal to the autocorrelation peak. Thus, the correlation is performedwithout loss in the signal-to-noise ratio. As mentioned hereinbefore,the coordinates of these cross-correlation peaks, as shown in FIG. 5,provides an indication of the scale difference, "a", and amount ofrotation between the two functions θ₀.

FIG. 4 shows a frequency plane correlator for forming the conjugateMellin transform M₁ * (ω.sub.ρ,ω.sub.θ) and for performing across-correlation operation utilizing a recording of this transform. Inapplicants' co-pending application, identified hereinbefore, there isdisclosed a procedure for producing a hologram corresponding to thisconjugate Mellin transform, and, as noted therein, the process involvesproducing at the input plane P₀, an image corresponding to the functionF₁ (expρ,θ). This image may be present on the target of an EALM tube asan appropriate transmittance pattern. Alternatively, it may be availableas a suitable transparency. In any event, the input function isilluminated with a coherent light beam from a laser source, not shown,and Fourier transformed by lens L₁. Its transform M₁ (ω.sub.ρ,ω.sub.θ)is interferred with a plane reference wave which is incident at an angleΨ and the resultant light distribution pattern is recorded. One of thefour terms recorded at plane P₁ will be proportional to M₁*(ω.sub.ρ,ω.sub.θ).

In carrying out the correlation, the reference beam is blocked out ofthe system. The hologram corresponding to the conjugate Mellin transformM₁ *(ω.sub.ρ,ω.sub.θ) is positioned in the back focal plane of lens L₁at plane P₁. The input image present at plane P₀ now corresponds to thefunction F₂ (expρ,θ) which again may be the transmittance pattern on anEALM tube or suitable transparency. When the coherent light beamilluminates the input plane P₀, the light distribution incident on planeP₁ is M₂ (ω.sub.ρ,ω.sub.θ). One term in the distribution leaving planeP₁ will, therefore, be M₂ M₁,* and the Fourier transform of this productis accomplished by lens L₂. In the output plane P₂, as shown in FIG. 5,two cross-correlation peaks occur. Two photodetectors spaced by 2π maybe utilized to detect these peaks and, as indicated hereinbefore, thesum of their amplitudes will be equal to the autocorrelation peakproduced when the two images being compared have the same position,scale and orientation.

In the correlation method depicted in FIG. 3, the conjugate Mellintransform M₁ *(ω.sub.ρ,ω.sub.θ) was produced and utilized in thefrequency plane of the correlator of FIG. 4. However, it should beunderstood that the method can also be practiced by utilizing theconjugate Mellin transform M₂ *(ω.sub.ρ,ω.sub.θ) at P₁ forming theproduct M₁ M₂ * and Fourier transforming it to complete thecross-correlation process.

What is claimed is:
 1. A method for correlating two functions f₁ (x,y)and f₂ (x,y) which are scaled and rotated versions of each other,comprising the steps ofobtaining |F₁ (ω_(x),ω_(y))|, and |F₂(ω_(x),ω_(y))|, the magnitudes of the Fourier transforms of thesefunctions; performing a polar coordinate conversion on |F₁(ω_(x),ω_(y))| and |F₂ (ω_(x),ω_(y))| thereby to obtain the functions F₁(r,θ) and F₂ (r,θ); logarithmically scaling the coordinate r in thefunctions F₁ (r,θ) and F₂ (r,θ) thereby to obtain the functions F₁(e.sup.ρ,θ) and F₂ (e.sup.ρ,θ); Fourier transforming F₁ (e.sup.ρ,θ) andF₂ (e.sup.ρ,θ) thereby to obtain the Mellin transforms M₁(ω.sub.ρ,ω.sub.θ) and M₂ (ω.sub.ρ,ω.sub.θ); obtaining the conjugateMellin transform M₁ *(ω.sub.ρ,ω.sub.θ); producing the product M₁ *M₂ ;Fourier transforming said product, all of the aforementioned steps beingperformed by optical or electro-optical means; and recording on film theresults of said last-mentioned Fourier transformation.
 2. In a method asdefined in claim 1 wherein said function f₁ (x,y) is in the form of anoptical transmittance pattern and the magnitude of the Fourier transformof this function F₁ (ω_(x),ω_(y)) is obtained by positioning saidpattern in the front focal plane of a lens and illuminating said patternwith coherent light whereby the light distribution pattern in the backfocal plane of said lens corresponds to Fourier transformation of saidfunction.
 3. In a method as defined in claim 1 wherein the conjugateMellin transform M₁ *(ω.sub.ρ,ω.sub.θ) is recorded as a filmtransparency of the interference pattern between a reference plane waveand a light distribution pattern corresponding to F₁ (e.sup.ρ,θ).
 4. Ina method as defined in claim 1 wherein the product M₁ *M₂ is produced bypositioning said film transparency having M₁ *(ω.sub.ρ,ω.sub.θ) recordedthereon in the front focal plane of a lens and illuminating said filmwith a light distribution pattern produced by Fourier transforming F₂(e.sup.ρ,θ).
 5. In a method as defined in claim 1 wherein the conjugateMellin transform M₂ *(ω.sub.ρ,ω.sub.θ) is obtained by having a referencewave interfere with a Fourier transformation of F₂ (e.sup.ρ,ω), theresultant interference pattern containing a term corresponding to saidconjugate.
 6. In a method as defined in claim 1 wherein the product M₁M₂ * is obtained by illuminating a film transparency having M₂ *recorded therein with the light distribution pattern resulting fromFourier transforming F₁ (e.sup.ρ,θ).
 7. A method for correlating twofunctions f₁ (x,y) and f₂ (x,y) which are scaled and rotated versions ofeach other, comprising the steps ofproducing an optical representationof |F₂ (ω_(x),ω_(y))|, the magnitude of the Fourier transform of f₂(x,y); performing a polar coordinate conversion on this representationof |F₂ (ω_(x),ω_(y))| thereby to obtain an optical representation of thefunction F₂ (r,θ); logarithmically scaling the coordinate r in thefunction F₂ (r,θ) thereby to obtain an optical representation of thefunction F₂ (e.sup.ρ,θ); Fourier transforming F₂ (e.sup.ρ,θ) thereby toobtain a light distribution pattern corresponding to the Mellintransform M₂ (ω.sub.ρ,ω.sub.θ); pouring a film transparency having aninterference pattern recorded therein which contain a term that isproportional to the conjugate Mellin transform M₁ *(ω.sub.ρ,ω.sub.θ);illuminating said film transparency with said light distribution patternthereby to produce a light distribution pattern corresponding to theproduce M₁ *M₂ ; Fourier transforming said last-mentioned light pattern;and recording the results thereof.
 8. Apparatus for correlating twofunctions f₁ (x,y) and f₂ (x,y) which are shifted, scaled and rotatedversions of each other, comprising in combinationan optical correlatorhaving an input plane P₀, a frequency plane P₁ and an output plane P₂ ;a film transparency having an interference pattern recorded thereinwhich contain a term proportional to the conjugate Mellin transform M₂*(ω.sub.ρ,ω.sub.θ), said film transparency being positioned at plane P₁; means for creating a light pattern leaving plane P₀ that correspondsto F₁ (e.sup.ρ,θ),said light pattern being Fourier transformed by lensmeans within said correlator located between planes P₀ and P₁ and theillumination of said film transparency by the resultant light patternproducing a light pattern leaving plane P₁ that corresponds to theproduct M₂ *(ω.sub.ρ,ω.sub.θ) M₁ (ω.sub.ρ,ω.sub.θ),said last-mentionedlight pattern being Fourier transformed by other lens means within saidcorrelator located between planes P₁ and P₂ ; and means positioned atplane P₂ for recording the results of said last-mentioned Fouriertransformation.
 9. Apparatus for correlating two functions f₁ (x,y) andf₂ (x,y) which are shifted, scaled and rotated versions of each other,comprisingoptical means for Fourier transforming f₁ (x,y) so as toobtain |F₁ (ω_(x),ω_(y))|, the magnitude of the Fourier transform ofthis function; means for performing a polar coordinate conversion on |F₁(ω_(x),ω_(y))| so as to obtain the function F₁ (r,θ); means forlogarithmically scaling the r coordinate in the function F₁ (r,θ) so asto obtain F₁ (e.sup.ρ,θ), |F₁ (ω_(x),ω_(y))|, F₁ (r,θ) and F₁(e.sup.ρ,θ) occurring as optical images; optical means for Fouriertransforming F₁ (e.sup.ρ,θ) so as to obtain a light patterncorresponding to the Mellin transform M₁ (ω.sub.ρ,ω.sub.θ); a filmtransparency having an interference pattern recorded therein whichcontains an optical representation of the conjugate Mellin transform M₂*(ω.sub.ρ,ω.sub.θ); means for illuminating said film transparency withsaid light pattern so as to obtain a light pattern corresponding to M₁M₂ *; optical means for Fourier transforming said last-mentioned lightpattern; and means for recording the results thereof.
 10. The apparatusas defined in claim 9 wherein said means for creating a light patternleaving plane P₀ that corresponds to F₁ (e.sup.ρ,θ) includesmeans forFourier transforming an optical representation of the function f₁ (x,y)so as to obtain |F₁ (ω_(x),ω_(y))|, the magnitude of the Fouriertransform of this function; means for performing a polar coordinateconversion on the light pattern resulting from said transformation so asto obtain a light pattern cooresponding to the function F₁ (r,θ); andmeans for logarithmically scaling the r coordinate in saidlast-mentioned light pattern thereby to obtain a light patterncorresponding to F₁ (e.sup.ρ,θ).